What Kind of Information is Necessary for NLP and MT?

Researchers in natural language processing (NLP) and machine translation (MT) in the past were mainly interested in linguistic theories, parsing and generation. They discussed specific types of language expressions such as garden path sentences, and did not pay much attention to problems v0hich may arise when huge volmne of real existing sentences are handled, such as newspaper articles, patent documents, and so on. Once we get into this area of processing large text corpus, we confront with several problems such as: we have to write a complete set of grammatical rules, we have to prepare a comprehensive dictionary, and so on. A major probleln here is not in the linguistically interesting but seldom-appearing linguistic phenomena, but in the average success rate of parsing, generation etc. for a large text corpus, which seldom includes such sophisticated sentential structures as garden path sentences. It includes different types of difficult problems, for example, parsing of long sentences such as sentences COlnposed of more than thirty words, and building a good lexicon. In the university researches where new idea is most important, building a good complete lexicon has not been a central issue. At companies on the contrary where COlnmercial machine translation systems are commercialized, people are forced to construct a complete dictionary which includes all the common words and sufficient number of terminology words in several specific fields. Dictionary construction takes a long time and a big money. It has to have a consistency in the whole dictionary, and the quality of the contents must be uniform for all the words. Japanese companies spent a lot of money for the dictionaries for machine translation. From this bitter experience they got together to construct a basic electronic dictionary which they can share themselves by the help of the Japanese Government.